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The  Joint  Synthetic  Test  and  Evaluation  Battlespace  (JSTEB)  is  the  permanent  infrastructure 
needed  to  support  routine  distributed  test  and  evaluation.  The  JSTEB  is  comprised,  of  networks, 
common  data  standards  and  protocols,  integrated  scheduling,  configuration  and  mission  control. 
It  facilitates  the  accessibility  of  live,  virtual,  and  constructive  models  and  simulations  from 
widely  dispersed  locations.  JSTEB  is  usable  by  the  Acquisition,  Test  and  Evaluation,  Training, 
and  Operations  communities.  Most  importantly,  it  enables  the  early  and  continuous  involvement 
of  the  warfighter,  user,  developer,  and  tester  in  the  acquisition  process.  A  synthetic  battlespace 
is  needed  to  facilitate  Simulation  Based  Acquisition,  implement  the' Simulation,  Test,  and 
Evaluation  Process,  perform  comprehensive  test  and  evaluation,  and  assess  joint  warfighting 
capabilities.  The  understanding  of  weapon  systems  and  their  capabilities  gained  as  a  result  of 
using  a  synthetic  battlespace  for  test  and  evaluation  will  enhance  the  fundamental  understanding 
of  warfighting  systems  and  make  it  easier  to  optimize  designs  for  performance,  producibility,  and 
affordability. 


Test  and  evaluation  challenges 

Comprehensive  test  and  evaluation.  The  Honorable  Jacques  S.  Gansler,  Under  Secretary  for 
Acquisition  and  Technology,  in  his  keynote  address  at  the  National  Defense  Industrial 
Association’s  Simulation-Based  Acquisition  Workshop  held  in  Orlando,  Florida,  on  March  17, 
1998  said,  “We  must  also  make  far  greater  use  of  modeling  and  simulation  in  our  test  and 
evaluation  process.  We  must  actually  replace  some  of  our  testing  with  simulation  and  do  so  in 
ways  that  actually  improve  the  quality  of  the  process.” 

The  address  further  pointed  out  that  “simulations  are  of  extremely  limited  value  if  they  are  not 
validated  by  realistic  system  and  subsystem  testing.”  The  speech  went  on  to  confinn  Aat 
comprehensive  testing  is  not  done  today  with  the  statement,  “testing  itself  is  of  extremely  limited 
value  considering  the  very  few  data  points  obtained  with  exceptionally  high  cost  of  modem 
weapons,  and  the  vast  array  of  possible  test  conditions.”  (Gansler  1998)  The  limited  value  of 
testing  can  be  greatly  enhanced  through  the  use  of  credible  simulations. 

Simulation-Based  Acquisition/Simulation,  Test  and  Evaluation  Process.  The  competing 
factors  of  reduced  funding  versus  increased  readiness  have  forced  the  Department  of  Defense 
(DoD)  to  'look  for  strategies  to  build  more  technologically  sophisticated  systems  cheaper  and 
faster  than  ever  before.  Simulation-Based  Acquisition  (SBA)  is  DoD ’s  vision  of  an  acquisition 
process  that  dramatically  reduces  system  life  cycle  costs  while  increasing  the  quality,  military 
utility  and  supportability  of  systems  developed  and  fielded.  The  intent  of  SBA,  however,  can 
only  be  achieved  if  a  system  is  originally  designed  to  cost  less  to  operate  and  maintain.  Thus, 
SBA  is  enabled  by  state-of-the-art  simulation  and  data  sharing  technology  integrated  across 
acquisition  phases  and  programs  to  provide  comprehensive  and  reliable  data  for  measuring 


program  costs  and  performance  results  and  making  well-informed  decisions  before  life  cycle 
costs  are  effectively  rendered  unchangeable.  Acquisition  program  managers  must  plan  for  the 
use  of  modeling  and  simulation  (M&S)  in  their  acquisition  strategies  and  make  the  up-front 
investment  in  the  necessary  applications  and  technology  to  make  SB  A  a  reality. 

A  key  element  of  SBA  is  the  Simulation,  Test,  and  Evaluation  Process  (STEP)  which 
significantly  reengineers  the  way  models  and  simulations  are  used  in  the  test  and  evaluation 
(T&E)  process.  STEP  more  effectively  integrates  simulation  into  the  T&E  process  for  the 
purpose  of  iteratively  evaluating  and  improving  the  design,  performance,  joint  military  worth, 
survivability,  suitability,  and  effectiveness  of  systems  to  be  acquired.  The  product  of  STEP  is 
consistent  and  comparable  performance  information  that  the  requirements,  acquisition,  T&E,  and 
training  communities  can  apply  throughout  a  system’s  life  cycle.  By  beginning  the  process  early, 
models  and  simulations  can  provide  clear  and  up-firont  information  on  critical  design  aspects. 
This  allows  for  multiple  design  iterations  before  hardware  is  available  for  test.  STEP  results  in  an 
efficiently  designed  and  thoroughly  understood  system  and  provides  developers  and  testers  high 
confidence  in  its  operational  effectiveness,  suitability  and  survivability. 

On  October  3,  1995,  Dr.  Paul  Kaminski,  then  Under  Secretary  of  Defense  for  Acquisition  and 
Technology,  wrote  that  he  was  requiring  STEP  be  an  integral  part  of  Test  and  Evaluation  Master 
Plans  (TEMPs).  STEP  Guidelines  were  distributed  in  December  1997  and  are  expected  to  be 
reflected  in  test  planning,  conduct  and  TEMPs. 

Joint  Vision  2010.  Joint  Vision  2010  provides  a  conceptual  template  of  how  U.S.  Armed 
Forces  will  fight  in  the  21*‘  Century  and  a  template  to  guide  the  transformation  of  these 
warfighting  concepts  into  joint  operational  capabilities.  Joint  Vision  2010  describes  the  changing 
and  uncertain  strategic  and  operational  environment  in  which  our  Armed  Forces  will  be  expected 
to  achieve  full  spectrum  dominance.  Military  success  in  this  environment  will  depend  upon  the 
quality  of  our  fighting  forces  enabled  by  enhanced  jointness,  information  superiority, 
technological  advances,  enhanced  multinational  operations,  and  our  ability  to  understand  and 
adapt  to  changing,  potential  adversaries. 

Just  as  Joint  Vision  2010  provides  a  template  to  guide  the  transformation  of  warfighting 
concepts  into  joint  operational  capabilities,  it  also  provides  a  template  to  guide  the 
transformation  of  T&E  capabilities  to  “test  the  way  we  fight”  for  assessing  the  warfighting 
capabilities  of  future  systems. 

Needed  Capability 

Acquiring  systems  with  joint  warfighting  capabilities  imposes  significant  demands  on  the 
acquisition  process.  Specifically,  the  variety  and  scale  of  joint  task  force  scenarios  make  it 
impractical  to  consider  staging  such  events  as  the  principle  means  of  testing  the  efficacy  of 
newly  developed  or  upgraded  systems.  Through  the  use  of  advanced  distributed  simulation 
(ADS)  techniques,  however,  synthetic  environments  could  be  created  to  serve  in  place  of  large- 
scale  joint  task  force  scenarios  to  test  the  effectiveness  of  systems  in  joint  operations.  In  this 
context,  synthetic  means  anything  short  of  war.  Therefore,  the  Synthetic  environment  used  to  test 
a  new  system  could  consist  of  an  assembly  of  hve,  hardware-in-the-loop,  man-in-the-loop  and 
digital  simulations  networked  over  geographically  dispersed  locations  and  configured  to 
represent  the  desired  operational  or  test  environment.  These  synthetic  environments  would  not 
replace  live  tests,  but  would  be  used  to  supplement  them  or  extend  system  assessments  to  areas 
not  feasibly  evaluated  using  live  tests.  Furthermore,  live  testing  is  essential  in  verifying  the 
performance  and  credibility  of  synthetic  environments. 


Similarly,  constraints  on  testing  such  as  safety,  regulatory,  inadequate  test  assets,  lack  of 
representative  environment,  lack  of  operational  realism,  cost,  treaty,  security  and  limitations  on 
test  instrumentation  preclude  comprehensive  evaluations  based  on  live  testing  alone.  Again, 
synthetic  environments  used  to  complement  live  testirig  will  provide  decision-makers  with 
needed  information  which  otherwise  would  not  be  available. 

The  traditional  acquisition  process,  build-test-fix-test...,  depends  on  iterations  of  hardware 
builds  to  develop  the  resulting  operational  capability.  From  a  testing  perspective,  developmental 
and  operational  testing  is  pushed  towards  the  final  phases  of  hardware  development  where 
opportunities  for  change  are  few  and  expensive.  The  STEP  or  model-test-model. .  .-build  process 
is  achieved  by  using  virtual  prototypes,  in  lieu  of  physical  prototypes,  for  T&E  of  specific 
system  characteristics.  Virtual  prototypes  afford  the  following  advantages,  they  can  be:  1) 
produced  quickly  and  at  a  relatively  low  cost;  2)  endowed  with  characteristics  for  which 
technological  solutions  are  not  at  hand  and  without  the  expense  of  achieving  the  technical 
solution;  and  3)  operated  by  the  user  or  tester  in  advance  of  committing  funds  and  large 
development  programs  to  produce  a  physical  test  article.  Synthetic  environments  are  required 
for  engineers,  users,  and  testers  to  operate  virtual  prototypes  of  weapon  systems  and  benefit  firom 
the  understanding  of  their  capabilities  gained  as  a  result  of  using  STEP.  Likewise,  these 
S3mthetic  environments  would  consist  of  simulations  of  components  of  an  actual  combat 
environment,  configured  to  emphasize  system  and  subsystem  optimization  and  integration  and 
allow  early  and  continuous  involvement  of  warfighters,  developers  and  testers. 

Assessment  of  existing  T&E  capabiiities 

Many  of  the  sophisticated  simulations  that  could  be  used  in  constructing  synthetic  environments 
already  exist.  Test  and  training  ranges,  laboratories  and  facilities  have  developed  unique  assets 
which  could  make  invaluable  contributions  in  synthesizing  “battlefields.”  Test  and  training 
capabilities,  however,  have  autonomously  evolved,  resulting  in  duplication  of  effort  and 
resources,  and  differing  processes  and  procedures,  which  limit  the  interoperability,  sharing,  and 
reuse  of  resources  demanded  by  sjmthetic  environments. 

The  extensions  or  enhancements  to  test  and  training  infirastructure  necessary  to  accommodate 
assembly  and  use  of  synthetic  environments  are  a  prudent  investment.  As  stated  by  Senator  John 
McCain  in  his  address  to  the  Senate  regarding  American  military  readiness:  “Long  experience 
has  shown  us  that  even  the  best  test  and  evaluation  efforts  can  leave  equipment  with  significant 
problems  that  are  only  exposed  in  combat.  Comprehensive  test  and  evaluation  is  expensive,  and 
it  is  tempting  to  cut  comers.  Any  reduction,  however,  means  a  loss  of  readiness.”  (McCain 
1993)  Synthetic  environments  are  critical  to  expose  and  correct  significant  problems  prior  to 
combat. 

Joint  synthetic  test  and  evaluation  battlespace 

The  Joint  Synthetic  Test  and  Evaluation  Battlespace  (JSTEB)  is  the  capability  to  routinely  and 
cost  effectively  assemble  and  use  synthetic  environments  for  the  primary  purpose  of  T&E. 
Figure  1  illustrates  the  operational  concept  of  the  JSTEB. 
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Figure  1.  JSTEB  operational  concept 

The  JSTEB  builds  upon  proven  T&E  resources  and  capabilities,  makes  optimal  use  of  limited 
and  expensive  test  and  training  assets,  and  integrates  them  with  new  technologies  to  create  the 
synthetic  environments  needed  to  augment  live  testing.  The  JSTEB  responds  to  both  long¬ 
standing  and  emerging  challenges  in  the  test  community  with  a  framework  that  enables 
interoperability,  reuse,  and  sharing  of  models,  simulations,  and  data.  It  benefits  other 
communities  and  fimctions  as  well,  for  example,  in  training,  it  allows  commanders  to  see  the 
battle  at  different  levels  of  detail  or  increases  the  possibilities  for  innovative  or  enhanced  training 
(Figure  2). 


Figure  2.  JSTEB  functional  concept 


Funding  and  implementation  strategy 

The  concept  of  the  JSTEB  was  first  disseminated  in  May  1997  with  the  publication  of  the  Navy 
Test  and  Evaluation  Modeling  and  Simulation  Master  Plan.  The  Test  and  Evaluation  Modeling 
and  Simulation  Management  Office  (TEMSMO),  chartered  by  the  Director  for  Test  and 
Evaluation  and  Technology  Requirements  (N091),  and  located  at  the  Naval  Air  Warfare  Center 
(NAWC)  Aircraft  Division  (AD),  Patuxent  River,  Maryland,  has  the  responsibility  for  execution 
of  the  master  plan.  The  TEMSMO  has  adopted  a  strategy  of  incrementally  building  JSTEB  on  a 
least  cost  basis  as  a  cost-effective  means  of  improving  the  long-range  resources  of  the  T&E 
community.  Leveraging  other  M&S  initiatives  and  programs  to  provide  ADS  and  analytical  tools 
and  technology  for  JSTEB  is  also  included  as  part  of  the  strategy. 

The  TEMSMO  has  set  into  motion  a  number  of  initiatives  to  make  JSTEB  a  reality.  The 
charting  of  these  initiatives,  in  conjunction  with  the  leveraging  of  other  programs,  as  an 
implementation  strategy  is  illustrated  in  Figure  3. 


Figure  3.  JSTEB  implementation  strategy 

Pilots.  We  can  no  longer  afford  to  build  things  simply  because  they  are  good  ideas.  Pilot 
programs  are  focused  experiments  to  demonstrate  the  utility  and  cost/benefit  of  synthetic 
environments  supporting  T&E  while  developing  specific  portions  of  JSTEB.  Pilot  programs  also 
afford  the  Navy  the  means  to  take  a  measured  approach  in  creating  JSTEB  before  significant 
dollars  are  spent. 

Navy  T&E  ciurently  has  two  pilot  programs:  Integrated  Ship  Defense  (ISD)  Modeling  and 
Simulation  Pilot  Program  and  the  Joint  Strike  Fighter  (JSF)  Program: 

Integrated  Ship  Defense  M&S  Pilot  Program.  The  ISD  M&S  Pilot  Program  goal  is  to 
develop  and  demonstrate  a  comprehensive  modeling  and  simulation  capability  in  support  of  the 
design  and  evaluation  of  components  and  systems  directed  towards  phases  of  the  ship  defense 
mission,  i.e.,  detect,  control,  and  engage.  The  objective  is  to  utilize  ADS  technology  to  support 
the  acquisition  of  Navy  systems  using  SBA  initiatives  and  emerging  M&S  technologies. 


Interfaces  to  battlegroups,  joint  warfare  simulations,  and  natural  environment  simulations  will  be 
developed  to  improve  hard  kill  and  electronic  warfare  assessment  capabilities.  In  addition,  the 
pilot  will  establish  network  technology  for  interoperability,  determine  the  cost  and  technical 
challenges  associated  with  becoming  High  Level  Architecture  (HLA)-compliant,  and  determine 
ways  to  use  M&S  to  reduce  limitations  to  scope  of  testing. 

ISD’s  contributions  to  JSTEB  include  secure  networks  and  basic  federation  infrastructure  for 
reuse  at  the  Integration  Test  Facility  located  at  the  MITRE  Corporation  in  Reston,  Virginia  and 
an  HLA-compliant  federate  of  existing  models. 

Joint  Strike  Fighter  Program.  A  key  objective  of  the  JSF  Program  is  to  ensure  that  a  robust, 
“core  suite”  of  models,  simulations  and  supporting  databases  are  available  for  Government  and 
Industry  to  leverage  mutual  investments  and  provide  a  common  frame  of  reference  for  analysis. 
The  JSF  Program  refers  to  this  robust,  “core  suite”  of  models,  simulations  and  supporting 
databases  as  the  Virtual  Strike  Warfare  Environment  (VSWE).  The  VSWE  extends  beyond 
single  facilities  through  the  use  of  ADS  networks  and  protocols  that  allow  for  the  seamless 
integration  of  constructive,  virtual  and  live  simulations  among  geographically  separated 
facilities.  VSWE  M&S  will  comply  with  emerging  DoD  standards  such  as  the  HLA,  the 
Defense  Information  Infrastructure/Common  Operating  Environment,  and  the  Global  Command 
and  Control  System,  to  reduce  the  cost,  time  and  risk  of  integrating  the  JSF  and  its  M&S  support 
systems  into  a  Joint  system-of-systems  architecture. 

JSF’s  contribution  to  JSTEB  is  open-systems  software  that  supports  the  integration  of 
Maimed  Flight  Simulators  in  the  Air  Combat  Environment  Test  and  Evaluation  Facility,  NAWC- 
AD,  Patuxent  River,  Maryland  with  the  Manned  Flight  Simulator  in  the  Virtual  Prototyping 
Facility,  NAWC-Weapons  Division,  China  Lake,  California. 

TEAMS.  Because  of  the  independent  nature  of  acquisition  programs,  there  is  little  synergy 
among  them  today  on  common  M&S  and  T&E  issues.  The  Test  and  Evaluation/Acquisition 
Alliance  for  Modeling  and  Simulation  (TEAMS)  is  a  consortium  of  acquisition  and  T&E 
professionals  that  come  together  to  address  common  M&S  and  T&E  issues  and  benefit  from  any 
synergies  found.  Specifically,  TEAMS  addresses  standards  and  common  infrastructure  needs 
associated  with  assembling  and  using  synthetic  environments  and  other  issues  associated  with 
the  integrated  use  of  modeling  and  simulation  across  assessment,  design,  development,  and  T&E. 
TEAMS  ensure  program  requirements  are  considered  in  determining  JSTEB  development 
priorities,  and  ensures  a  JSTEB  architecture  is  developed  that  is  widely  understood  and 
compatible  with  a  broad  range  of  user  requirements.  This  is  essential  for  the  establishment, 
growth  and  evolution  of  JSTEB.  TEAMS  products  include  a  roadmap  to  guide  the  development 
of  JSTEB  and  pilot  projects  that  implement  specific  portions  of  the  roadmap.  The  TEAMS 
process  is  illustrated  in  Figure  4. 
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Figure  4.  T&.E/ Acquisition  AiUance  for  M&S  - TEAMS -process 


Policy.  Once  risk  reduction  is  completed  and  the  cost/benefit  of  using  synthetic  environments 
for  T&E  has  been  fully  demonstrated,  policy  for  TEMPs  will  be  addressed  to  ensure  acquisition 
programs  take  advantage  of  the  increased  capability  to  assess  the  performance  of  military 
systems  and  provide  substantial  cost  contaimnent  to  programs.  T&E  investments  will  be 
addressed  to  institutionalize  immediate  and  future  improvements  in  T&E  resources  necessary  to 
meet  the  demand  for  the  JSTEB  capability.  Currently,  policy  is  being  considered  to  stimulate 
and  enhance  M&S  reusability  by  requiring  all  developers  and  authors  of  T&E  models  and 
simulations  register  their  models  and  simulations  in  the  Navy  Test  and  Evaluation  Repository  for 
Models  and  Simulations  (NTERMS). 

Workshops.  Navy  TEMS  Workshops  have  been  critical  in  informing  the  T&E  community 
about  TEMS  initiatives,  gaining  support  for  those  initiatives,  and  establishing  requirements  for 
interoperability  and  reuse  of  T&E  M&S.  There  have  been  four  workshops  to  date  with  the  first 
workshop  held  in  F ebruary  1 994. 

During  the  JSTEB  development  process.  Navy  T&E  will  be  coordinating  JSTEB 
requirements  with  the  M&S  programs  and  initiatives  of  the  Office  of  the  Secretary  of  Defense 
(OSD),  Army,  Air  Force,  and  the  Marine  Corps.  This  will  ensure  interoperability  and  reuse  with 
other  Service  M&S.  A  Multi-Service  T&E  M&S  Workshop  is  planned  for  6-8  April  1999  to 
address  joint  issues  for  JSTEB.  The  workshop  is  entitled,  “The  Joint  Synthetic  Test  and 
Evaluation  Battlespace  —  Meeting  the  Warfighter’s  Needs.” 

Joint  Advanced  Distributed  Simulation.  The  Joint  Advanced  Distributed  Simulation 
(JADS)  Joint  Test  and  Evaluation,  sponsored  by  the  OSD  Joint  Test  and  Evaluation  Program  is  a 
“test”  of  ADS  applied  to  T&E.  JADS  legacy  products  will  be  incorporated  into  the  JSTEB 
architecture. 


Summary 

Constraints  on  testing  preclude  comprehensive  evaluations  based  on  live  testing  alone.  Synthetic 
environments  consisting  of  live,  hardware-in-the-loop,  man-in-the-loop,  and  digital  simulations 
can  augment  or  complement  live  testing  to  cost  effectively  provide  decision-makers  with  needed 
information  which  otherwise  would  not  be  available.  Synthetic  environments  support  a  changing 
acquisition  process  by  enabling  the  integration  of  the  T&E  process  across  a  product’s  life  cycle. 
Synthetic  environments  will  also  be  critical  in  assessing  foture  Service  and  joint  warfighting 
capabilities.  In  the  future,  the  T&E  community  will  use  JSTEB  to  routinely  and  cost  effectively 
assemble  and  use  synthetic  environments  to  perform  comprehensive  T&E,  achieve  the  objectives 
of  SB  A  and  STEP  and  to  “test  the  way  we  fight”  when  assessing  future  warfighting  capabilities. 
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Endnotes 

1 .  The  views  expressed  in  this  article  are  those  of  the  author  and  do  not  represent  the  official 
view  of  the  United  States  Department  of  Defense  or  the  United  States  Navy. 

2  .  Additional  information  concerning  the  Joint  Synthetic  Test  and  Evaluation  Battlespace  is 
available  from  the  Navy  Test  and  Evaluation  Modeling  and  Simulation  World  Wide  Web  site  at 
http  ://www.nawcad.navy.mil/tems. 
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